A Multimodal Machine Learning Approach for Scrub Typhus Classification Using Clinical and Skin Image Data

Abstract

Scrub Typhus is a vector-borne infectious disease caused by Orientia tsutsugamushi, transmitted through infected chigger mite bites, and is widely prevalent in tropical and subtropical regions, particularly in the Asia-Pacific zone. Early and accurate diagnosis is critical, yet challenging due to overlapping symptoms with other febrile illnesses such as dengue, malaria, and typhoid. Traditional diagnostic methods relying on clinical observation and laboratory tests are often time-consuming and prone to misdiagnosis. This paper proposes a multimodal machine learning framework that combines clinical symptom data with skin lesion image analysis for scrub typhus classification. Clinical features including fever, headache, myalgia, eschar, chills, sweating, and vomiting are analyzed using Support Vector Classifier (SVC) and Random Forest algorithms. Concurrently, a Convolutional Neural Network (CNN) based on the ResNet-18 architecture is applied to detect eschar patterns in skin lesion images. Explainable AI techniques, specifically SHAP (SHapley Additive Explanations) for clinical feature importance and Grad-CAM++ for image region visualization, are incorporated to improve model interpretability. The multimodal fusion of both data modalities achieves an overall accuracy of 0.75, precision of 0.74, recall of 0.73, F1-score of 0.73, and ROC-AUC of 0.78, outperforming unimodal approaches and existing comparative methods. The proposed system demonstrates the effectiveness of combining heterogeneous data modalities with explainable AI for early and reliable Scrub Typhus detection.

Introduction

Scrub typhus is a vector-borne infectious disease caused by Orientia tsutsugamushi, commonly found in the Asia-Pacific region, including India. It spreads through infected chigger mite bites and presents flu-like symptoms (fever, rash, headache, eschar), which often overlap with diseases like dengue, malaria, and typhoid, making early diagnosis difficult. Delayed detection can lead to severe complications and high mortality. Traditional diagnostic methods (ELISA, PCR, Weil-Felix) are accurate but require lab infrastructure, trained staff, and time, limiting their use in rural areas.

To address these issues, the work proposes a multimodal AI system that combines clinical symptom data and skin lesion images for improved diagnosis. Machine learning models (Random Forest, SVC) analyze clinical features, while a CNN processes images. The outputs are fused to produce a final classification (scrub typhus vs non-scrub typhus). Explainable AI methods like SHAP and Grad-CAM++ are used to improve interpretability.

The literature review shows that existing approaches typically rely on a single data source (clinical, image, or molecular), which limits accuracy. The proposed system fills this gap by integrating multiple data types.

The system pipeline includes data collection, preprocessing, model training, multimodal fusion, and explainable prediction. Evaluation using accuracy, precision, recall, F1-score, and ROC-AUC shows that the multimodal approach outperforms single-modality models, improving reliability and early detection capability.

Conclusion

This study presents a multimodal machine learning approach for scrub typhus detection using both clinical symptom data and skin lesion images. Machine learning algorithms are used to analyze clinical features, while a Convolutional Neural Network (CNN) is applied for image classification. The predictions from both models are combined to improve the accuracy and reliability of disease detection. Explainable AI techniques such as SHAP and Grad-CAM++ are used to interpret the model predictions. The results demonstrate that integrating multiple data sources can enhance scrub typhus detection and support healthcare professionals in early diagnosis and decision-making.

References

[1] Kala, D., & Sharma, R. (2020) – Reviewed different diagnostic methods for scrub typhus such as ELISA, PCR, and serological techniques. [2] Liao, C. C., et al. (2021) – Proposed an ELISA-based diagnostic platform for accurate scrub typhus detection. [3] Kim, M. G., et al. (2024) – Developed a molecular diagnostic platform to improve scrub typhus detection accuracy. [4] Lu, Y., et al. (2025) – Used machine learning techniques to predict the severity of scrub typhus using clinical data. [5] Kanchanapiboon, P., et al. (2024) – Applied deep learning and computer vision methods to analyze skin lesion images for scrub typhus detection. [6] [6] Li, F., et al. (2021) – Proposed a biosensor-based method for rapid detection of scrub typhus DNA.

Copyright

Copyright © 2026 Ms. Pasala Swathi Sri, Ms. Parimi Renuka Chowdary, Ms. Penke Naga Durga, Ms. Siraparapu Daiva Prasanna. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.